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1 Introduction. 



Among the latest fashions in nonparametric statistics are the so-called adaptive esti- 
mations (AE), i.e. estimations that use no apriory information about the estimated 
function. Many publications have recently appeared where AE are constructed which 
are optimal in order at a growing number of current observations on a continuum of 
various functional classes (cf. References for a list of works on AE, which does not, 
however, claim to be exhaustive). 

In (Polyak B., at al., 1990), (Polyak B. at al., 1992), (Golubev G. at al, 1992) for 
instance, AE were constructed for the problem of estimating regression (R) which 
are optimal in order on many subspaces of space L 2 , and non- adaptive confidence 
intervals were elaborated on the basis of the obtained estimations for the estimat- 
ed regression function also in norm L 2 , which later were somewhat improved in 
(Golubev at al, 1992). 

In (Efroimovich S., 1985) AE were constructed for problem (D) of estimating 
distribution density, which are optimal on ellipsoids in L 2 . 

In numerous publications by D. Donoho et al. (Donoho D at al., 1993(1), 1993(2), 
1996, 1999(1), 1999(2) ) and in some others AE are constructed (and implemented) 
which are optimal in order on a number of Besov spaces. In those papers as well as 
in (Golybev G. at al., 1994), (Nussbaum M., 1985), (Tony Cai at al., 1999), (Lee 
G., 2003) diverse orthonormalized systems of functions are used to construct AE, 
such as wavelets, wedgelets, unconditional bases, splines, Demmler - Reinsch bases, 
Ridgelets (Candes E.J., 2003), (Dette H., 2003) etc. 

The recent results about kernel estimations in the considered problems see, for 
example, (AAD W Van Der Vaart at al., 2003), Allal J., at al., 2003), (Corinne 
Berzin at al, 2003). 

In (Ostrovsky E.I., 1996, 1997(1); 1997(2), 1999) AE were constructed on the 
basis of the trigonometric approximation theory. 

In this work we construct AE based on the orthogonal polynomial 
expansion series - the Legendre polynomials. 

The AE proposed herein feature a speed of convergence which is op- 
timal in order on any regular subspace compactly embedded in space L 2 , 
the estimations are universal and very simple in form, which significantly 
facilitates their implementation; finally, we construct exponential adap- 
tive confidence intervals (ACI), i.e. such that the tail of the confidence 
probability decreases with exponential speed. 

2 Problem statement. Denotations. Conditions. 

R. Regression problem. Let f{x), x G [—1,1] be an unknown function, Riemann- 
integrable with a square and measured at points of the net Xi = x^ n = — 1 + 
2i/n, i — 1, 2, . . . , n\ n > 16 with random independent centered: E£j = identically 
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distributed errors {^} : yi = f(xi) It is required to estimate the function f(x) 
with the best possible precision from the values {yi}- 

D. Estimation of distribution density. On the basis of a set of independent 
identically distributed values & G [—1,1], i = 1,2, ...,n it is required to 

estimate their common density f(x) (assumed to exist). 

It is supposed that all the estimated functions /(•) G L 2 [— 1,1], therefore they 
are expanded in the norm of this space into a Fourier- Legendre series in the complete 
orthonormal system {Lj(-)} on the set [-1, 1]: 

oo ! 

f(x) = J2c j L j (x); Cj = Lj(x)f(x)dx, 

3=0 1 

where Lj(-), j = 0,1,2,... are normalized Legendres polynomials. The Legendre 
polynomials are given by the Rodrigues formula: 

1 d m 

P( x ) = " U X 2 - l) m ] 

with orthogonal property: 

I(k, m) = f ^ P m (x)P k (x)dx = 2 /(2m +1), m = k, 
otherwise I(k,m) = 0. We can define 



L k (x) = P k (x)Vk + 0.5. 

Let us set p(N) = p(f,N) = YfjLN+i ]- Evidently limjv^ooP(^) = 0. Let us also 
assume that only the non-trivial infinite- dimensional case will be considered, when 
an infinite multitude of Fourier coefficients / differs from zero, i.e. VW > 1 =>• 
p(N) > 0. Otherwise our estimations will converge in the sense L 2 (VL x [—1, 1]) with 
speed 1/y/n. 

Moreover, we assume further that (essentially infinite-dimensional case) 

lim \p(N)\/ log N = +oo. (2.0). 

iV^oo 

In other words, the condition (2.0) means that there exists the constant q G (0, 1) 
such that for all sufficiently great values iV =^ p(N) > q N . 

The value p(N) = p(f, N) is known and is well studied in the approximation 
theory. Namely, p(f,N) = E 2 N (f) 2 , where £'jv(/)p is the error of the best ap- 
proximation of / by the algebraic polynomials of power not exceeding TV in the L p 
distance: for g : [— 1, 1] — > R 1 we will denote 

\\9\\p=(f \g( x )\dx) ,p e [l,oo); Halloo = sup \g(x)\, 
\J-i / xe[-i,i] 

and closely connected with module of continuity of the form 
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to P ,2(f (k \t)= sup \fW( x + h)-2fW(x) + fW(x-h)\ p , 

h:\h\<5 



for instance: 



E n (f) P <C(p,r) n~ r u; Pt2 (f {r) A/n). 



[DeVore, Lorentz, p. 219-223]; arithmetical operations on the arguments of 
function / and their derivatives are understood as follows: ath>0x + h — 
min(x + h, 1), x — h = max(x — h, —1), h > 0. 

Everywhere below condition (7I) will be considered fulfilled: 

( 7 1) : IiW-+ooP(2A0MA0 = 7 < 1, (2.1) 
sometimes stronger conditions (7) as well: 

(7): 3 lim p{2N)/p{N) d d 7 < i; (2.2) 



( 7 0) : 7 = 0. (2.3) 
It is easy to show that it follows from condition (2.0) 

p(N) < CN~ 2 ^ 2(3 d ^ log 2 (l/ 7 ) > 0. (2.4) 

In the problem (R) it will be assumed that (3 > 1/2. There are some grounds to 
suppose that at (3 < 1/2 asymptotically optimal AE do not exists in the regression 
problem; for a similarly stated problem this was proved by Lepsky (Lepsky O., 
1990). 

Here and below the symbols C, C r will denote positive finite constructive con- 
stants inessential in this context, x is the usually symbol, in detail: 

A(n) x Bin) d < hmm£A(n)/B(n) < 

n— +00 

limsupA(n)/S(n) < C 2 , 3C U C 2 G (0,oo). 

n^oo 

the symbol A ~ B means that in the given concrete passage to the limit limA/B = 
1. 

Examples. Denote by W(C,a,(3) a class of functions {/} such that 

p(f, N) ~ CN- 2 P(\ogN) a , 3C,(3>0;a = const; 
W(a,(3) = U c> oW(C,a,(3); 

W((3) = W(0,(3); W = U/3 >0 W((3). 



5 



For the class of functions W the condition (7) is fulfilled. 
Also let us denote 

Z(a, /?) = {/: P (f, N)~ a (5 N }, a > 0, (5 G (0, 1); 

and Z = U a> o ; /3e(o,i)-^(ai, P). For the functions of class Z the condition (7O) is also 
fulfilled. Besides, functions of class Z are analytical. 

Denote for the problems R, D respectively at j < n dj = 

n n 

(i/n)5>M*i); c,- = (iA0E L ;(fc); (2.5) 
1=1 1=1 

j — 0,1,2,3, ... ,n — 1; and for the regression problem 

B(n,N)= c k (n) 2 + a 2 N/n; 

k=N+l 



a 2 = Var[£j]; for the problem (D) we define a 2 = 1 and 

2N 

B(n,N)= ]T 4 + N/n; 



k=N+l 

and again for both the considered problems set B{n) = 

min B{n, N), N° = N°(n) = argmin B(n,N); 

JV=l,2,...,[n./3] AT=l,2,...,[n/3] 

A(n, N) = p(N) + cr 2 N/n, A(n) = min A(n, N). 

JV=l,2,...,[n/3] 



For instance, suppose that / G W(C, a, f3), then A(n) 



n 



-2/3/(2/3+1) 



(logn) a// ^ 2/3+1 \ and in case / G Z(a,(3) =^ A(n) x logn/n. 



Our notation should not be surprising, as it follows from condition (7I) that all 
the introduced functionals {B(n, N)}, {B(n)} arising from different problems are 
mutually x equivalent. Besides, for the same reasons 

A(n, N) x B(n, N); A{n) x B(n). 

Apart from that it is clear that in the regression problem conditions must be 
imposed not only on the estimated function, but on the measurement errors £j too. 
We will consider here only the so-called exponential level. Indeed, we assume that 
in the regression problem the following condition is satisfied: 

(Rq) :3q,Qe (0, 00), =^ P(|&| > x) < exp (-(x/Q) q ) ,x>0. 

The so-called classical projective estimates was introduced by N.N.Tchentsov 
[Tchentsov N.N., 1972, p. 286] (for the trigonometrical system instead considered 
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here Legendre's polynomials will be considered as an estimates of the function 

/: 

N 

f(n,N,x) = Y,cM*)- (2-6) 

j=0 

Since, as is shown by Tchentsov, E||/(n, N, •)—/(•) || 2 x B(n,N), the selection of the 
number of summands iV optimal by order in the sense of L 2 (J1) x L 2 [— 1, 1] is given 
by the expression iV = N°(n) with the speed of convergence f(n, N°, •) — > /(•) in the 



above-mentioned sense is y A(n). I. A. Ibragimov and R. Z. Khasminsky (Ibragimov 
L, Khasminsky R., 1982) proved that no faster convergence exists on the regular 
classes of functions given by the value ^ ' A(n). 

However, the value p(f, N) or at least its order as A" — > oo are practically 
unknown as a rule. Below the adaptive estimation of / will be studied based only 
on the observations {^} and using no apriory information regarding /, and yet 
possessing the optimal speed of convergence at apparently weak restrictions. Set 

2N 

t(N) = r(n, N) d = ]T c 2 k , N(n) d = argmin r(n,N), (2.7) 

k=N+l Ne(l,[n/3]) 

T*(n) — min r(n,N), 

Ne(l,[n/m 

Our adaptive estimations f in both considering problems have a universal view: 

N(n) 

f = f(n,N{n),x)= £^0*0- (2.8) 

0=1 

In case of a non-unique number of summands N(n) in (2.7) we choose the largest. 
Below the value A^ will always be arbitrary non-random integer number in the set 
of integers numbers of the segment 1, 2, . . . , [n/3] and N(n) is the random variable 
defined in (2.7). 

Note, that by using the Fast Legendre Transform technique described by D. Potts 
et al. (D. Potts et al., 1998), the amount of elementary operations for / calculation 
is O (n log n) , likewise in Fast Fourier Transform and in Fast Wavelet Transform. 

Before proceeding to formulations and proofs let us clarify informally our idea 
for choosing N(n). It is easy to find by direct calculation for the regression problem 
(and analogously for the problem D) that the coefficients estimations have a view: 

4 = Cfc(n) + n~ 1/2 9 k (n), 

where asn — > oo 

Cfc(n) = n' 1 f(xi)L k (xi) -> / f(x)L k (x)dx = c k ; 

„■ — 1 J — 1 
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e k (n)=n- 1 ' 2 it l ZMx i ). 
i=i 

It follows from the multidimensional CLT that the variables {0 k (n)} as n — > oo 
are asymptotically Gaussian distributed and independent: 

Var[O k (n)] = n^fy^fo) - a 2 /' L 2 (x)^ = a 2 ; 
i=i 

n „x 

E0 fc (ra)0i(n) = <7 2 n _1 E L^x^L^Xi) -> ct 2 / L k )x)U(x) dx = 0, fc ^ Z. 

i=l 

Therefore, the variables {0/c(n)} are asymptotically independent and have approxi- 
mately the normal distribution: 

Law(c k ) x N(c k ,a 2 /n), 

or equally 

4 = Cfc + cre fe / Vn, Law(e k ) x 7V(0, 1) 
and also {e fc } are asymptotically independent. Therefore, r(n, iV) x 

2W 2N 2N 

E 4 + 2n-V^ ^ c^ + ^n" 1 E 4; 

fc=W+l fc=V+l k=N+l 

Er(n, AT) x E(n, N), Var[r(n, AT)] x 5(n, N)/n, (2.9) 

and therefore 

N-^oo, N/n -> =>• v /Var[r(n,A^)]/Er(n, A/") -> 0. 
Note that in the case of the regression problem the condition 

/3 > 1/2 (2.10) 

is essential which is common in statistical research (Polyak B. at al., 1990, 
1992),(Lepsky O., 1990). We will assume in the problem (R) that the condition 
(2.10) is satisfied. 

It follows from (2.9) that there are some grounds to conclude 

r(n, AO x Er(n, N) x A(n, N) 

and therefore 

N(n) = argminr(n, N) ~ argmin Er(n, N) = N°(n). 

N<n/3 ' N<n/3 

Also note that the number of summands N(n) proposed by us is a random variable (!) 
and that estimation (2.8) is non-linear by the totality of empirical Fourier coefficients 
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3 Formulation of the main results. 

Further we will investigate the exactness of our adaptive estimations in the L 2 sense 
in our interval [-1,1] and will write as usually 

\\f-g\\ 2 = f_(f(x)-g(x)f dx. 

Define also for the problem (R) 

r = r(q) = 2q/(q + 4), q e (0, 2); q > 2 r = q/(q + 1), (3.0) 

and r = 1 for the problem (D). 

Theorem R(q). Under the conditions (Rq) , (7I) , in the problem R we propose 
that there exists a constant Kr = Ku(q,^) e (0, 00) such that for the variable 

(r = ( R (n) = B~\n) Q' 2 \\f - f\\ 2 - Q~ 2 K R 
the following inequality holds: 

PGCftl > u) < 2exp (-C u r ' 2 (n A(n)) r ' 2 / log log n) , u > 1. (3.1) 

(See in comparison (Bobrov P. at al., 1997); here the exponent indices are sig- 
nificantly decreased.) 

Theorem D(q). Under the condition (7I), in the problem D we propose that 
there exists a constant K D = K D (q,~f) such that for the variable 

( D = ( D (n) = B- 1 (n)\\f-f\\ 2 -K D 
the following inequality holds: 

P(|Cd| > u) < 2exp (-u l/2 (n A(n)) l/2 / log log n) , u > 1. (3.2) 

This result improves the one for the Fourier approximation of (Ostrovsky E., 
Sirota L., 2004). 

Theorem (Rq) a.s. If in the problem (R) under condition (Rq) for arbitrary 
e > the series 

t> ^def ^ / (nA(n)y/ 2 \ , 00 . 

{n>16} {n>16} V 1U & 1U & ,A / 

converges, then in the sense of convergence with probability one 

lim r*(n)/B(n) = 1 (3.4a) 

and 

Iini n ^ 00 ||/-/|| 2 / J B(n) <K R . (3.4.6) 
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Let us make another additional assumption (v) with regard to the class of esti- 
mated functions {/}. Denote 

Bin, N) 

H n (v) = inf — 777—, — , v = const > 1; 
Ne[NO/ v ,No v ] B(n) 

pW(n,v) = exp (-{^[{H n {v)) - l) r/2 {nB{n)) r/2 / log log n 



(At v > N° the left interval is absent, at v > n/(3N°) thr right interval is absent.) 
Condition (v) : 

Vt; > 1 £ P (N) {n,v) < 00. 

n>16 

The classes of functions satisfying conditions (7I) and (v) will be called regular. 
Classes W and Z are regular. 

Corollary 1. If in addition to the conditions of theorem R(q) the condition (v) 
holds, then a.e. 

lim N(n)/N°(n) = 1. (3.5) 

Theorem (D) a.s. Let for problem (D) besides the above-formulated assump- 
tions, condition (3.3) also be fulfilled with r/2 replaced by 1/2. Then the factual 
convergences of (3.4 a) and (3.4-b) are asserted here as well. 

Corollary 2. Analogously if in addition to the our the condition (v) holds, then 
also with probability one 

lim N(n)/N°(n) = I. 

n—too 

4. Proofs. 

The proofs of the theorem (Rq) and (Dq) are similar to proofs of the our result 
in (Ostrovsky E., Sirota L., 2004) for trigonometrically approximation for f(x); we 
will use the known properties of Legendre's polynomials (Kallaev, 1970), (Szego, 
1959). For instance, 

sup sup \L k (x)\ (l — x 2 \ ^ < 00. 

k xe[-i,i] 

Instead of the semi invariant estimations for polynomials from independent random 
variables (Saulis, Statuliavitchius, 1989) we will use the modern estimations for 
polynomial martingales (Hall, Heyde, 1980) [pp. 115 - 120], (Ostrovsky. E, 2004). 

First of all we consider the problem of regression (R). 

We will assume without loss of generality Q — 1. 

STEP 1. Let us write the exact expression for the important variables. Intro- 
duce the notation: 
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S k (n) d = Cfc(n) - c fc . 



We can write: 



Ck(ra) = c fc + 5 fc (n) + n 1 E £i L k(%i), 

i=l 

{c k {n)f = c 2 k + S 2 k (n) + a 2 n- 2 £ L 2 k (x t ) + 

i=i 

n n 

2c k 5 k (n) + 2n _1 E c k ^L k (xi) + 2n _1 E ^(n)^L t (ij)+ 



i=l 



1=1 



71 



7t 



i=l 



We have for the variables t(N, n) (and further for the variables A 2 = A 2 (iV, n) 
\\f-f\\ 2 ):r(n,N) = 



2N 2N 2N 2N n 

E 4 + 2 E c k 5 k (n)+ E + ^n" 1 E ^E^ 

fe=V+l fc=N+l fc=N+l fc=V+l i=l 



+ 



2 ^ _1 E^ E h{n)L k (xi) + 2n _1 E^i E c kL k {xi) 

i=l k=N+l i=l fc=V+l 



+ r 2 , 



T 2 



2AT 



1 E (C 2 - - 2 ) E ^ 



i=l 



fc=V+l 



+ 



2N 



2n X E E n 1 E L k (xi)L k (xj) 

l<i<j<n k=N+l 



(4.1) 



where r = r + ri + r 2 ; r m = r m (n, iV), r is the deterministic part of r : Er = 
t ~ -B(n, iV), ri is the linear combination of r 2 is the bilinear combination of 

{&}■ 

It is easy to verify using the known properties of Legendre's polynomials that 
Ti x B(n, N) and that 

Var[n] < CB(n,N)/n, Var[r 2 ] < CB(n,N)/n. 

STEP 2. Note that the sequences 771(71) = £"=1 (i), ?7 2 (n) = £™ =1 - 
er 2 ) and 

^sH = E E KhMiCj, 

l<i<j<n 
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where {b(i)}, {b(i,j)} are a non-random sequences, with the second component 
F(n) = a ({&}, i = 1, 2, . . . , n) , i.e. {r} s (n), F(n)}, s = 1, 2, 3; {F(n)} is the natural 
sequence (flow) of sigma-algebras, are martingales. 

It follows from the main result of paper (Ostrovsky E., Sirota L., 2004), devoted 
to the exponential and moment estimations for martingale distributions, that 



n>16 



sup P(|ti )2 |/ JVar [r lj2 ] > x) < exp (-Cx r ) , x > 0. (4.2) 



STEP 3. Now we intend to use on the basis of inequality (4.2) the Law of 
Iterated Logarithm (LIL) for the martingales (Hall, Heyde, 1980) [pp. 115 - 121] 
in the more convenient for us form (Ostrovsky, 1999) [pp. 79 - 83]. Namely, if we 
denote 



v(n,N) = (r(n, N) - Er(n, N))/yjB(n, N)/n, 
then Ez/(n, N) = and if we denote 

( d — sup sup | u(n, N) | / log log n, 

n>16JV6[l,n/3] 

then ( < oo a.e. and for the random variable ( we have for all positive values 
x, x > the tail inequality 

P(C>x)<exp(-C(gy-). (4.3) 
The inequality (4.3) may be rewritten as follows: 

r{n, N) = Er(n, N) + ({n, N) log log n ^B{n,N)/n, (4.4) 

where ( = sup n Ar |C(n, N)\ satisfies the inequality (4.3), Er(n, N) = B(n, JV)(1 + 
0{n)), 6{n) is non-random and lim^oo 6*(n) = 0. 

STEP 4. Let M be some subset of an integer segment S — [1, 2, . . . , n], M — 
S\M, ?r(M) = J P(iV(n) G M), and assume that 

u = u(n,M) = f m£ M B(n,N)/B(n) > 1. 

Then under conditions (7) and (i?g) 

tt(M) < 2exp ^-C [(v 7 ^- 1) nB(n)] r/2 /(loglogn)) . (4.5) 
Proof. We obtain for the case of (Rq), denoting 77 = m&x NeS \ u(n, N)\ : 
vr(M) = P(N(n) E M) = P(min_ r(n,N) > min r(n, N)) = 



P min(B(n,N) + JB{n,N)/n (loglogn) 1/r v(n,N)) > 
\NeM J 
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mm \B(n,N) + y/B(n,N)/n (loglogn) 1 ^ u(n, N)j < 

P(B(n) + \J B{n)/n (loglogn) 1/r V > vB(n) - \JvB{n)/n (loglogn) 1/r V). 
We find solving the inequality under the probability symbol relative to 

V : tt(M) < P (v (1 + y/tyyjB(n)/n (loglogn) 1/r > (v - l)B(n) \ < 



( v -i JnB(n) \ ( JnBin) \ 

p p^(i4^j =P r- ( ^- 1) (i4^j' (46) 

Using our estimations (4.6) for r, we arrive to the assertion (4.5). 
Note that under our condition (2.0) 

nB(n) > C logn, C l G (0, oo), 
therefore under our conditions for all values v, v > 1 

lim tt(M) = 0. 

n^oo 

If in addition for any e > the series EnPn(s) converges, the assertions 
(3. 4. a), (3. 4. b) and corollaries 1,2 to be proved follows from the lemma of Borel- 
Cantelli. 

The rest is proved analogously if it is taken into account that N > N°(n)(l + 
e), e E (0,1] and condition (v) lead to the inequality B(n,N) > (1 + Ce 2 )B(n), 
e G (0, 1); this completes the proof. 

Analogously we can prove the theorem (Rq)a.s, on the basis of inequality: 

P n (e) < exp (-Ce r / 2 (nA(n) r / 2 )/(loglogn)) . 
Note in addition that at v > 2 

r*(n) < B(n)(l + 9(n))+V log logn y/B(n)/n, 
therefore we have for sufficiently great values n : 

P(\r*(n)/(B(n)(l + 9(n)))\ > v) < exp (-Ce r/2 (nA(n) r/2 )/(loglogn)) (4.7) 
and analogously 
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P(|r*(n)/(5(n)(l + 9(n)))\ < 1/v) < exp [-Ce r/2 (nA(n) r ' 2 ) / '(log log n)) . (4.8) 



STEP 5. Let us consider here the main variables A 2 . We have: A 2 



oo N(n) n N{n) N(n) 

E 4 + ^n' 1 E « _1 E ^) + E + 2 E c k Sk(n) 

k=N(n)+l k=0 i=l k=0 k=0 



+ 



n N n N 

2 ^ _1 E^ E h{n)L k (xi) + 2n _1 c k L k (xi 

i=l k=0 i=l k=0 



+ 



n 



N(n) N(n) 

1 E (c 2 - * 2 ) ^ E ^te) + sn- 1 E E n- 1 e M^te) 

l<i<j<n 



i=l 



fc=0 



fc=0 



A + Ai + A 2 . We have analogously to the investigation of the expression for 
r(n, N) using the condition 7 for sufficiently large values n > n > 2 : 

11/ - f\\ 2 /B(n) < CA(n,N(n))/B(n) + V 3 (N(n))/B(n) < 

C(l - 7 )- 1 r*(n)/ J B(n) + tt 3 (JV(ra))/£(n) = 

C(l - + (r*(n)/S(n) - 1) + * 3 (^W)/S(n), 

where, as can easily be seen, \l/ 3 (iV) = Ai + A 2 . 

Then we will use the elementary inequality P(A) < P(AB) + P(B), in which 
A, B are events. Setting A = 

{||/ - f\\ 2 /B(n) - C/(l - 7) > «}, B = {1/v < r*(n)/B(n) < v}, 

we have at v e (2, u — C) : 

P = 7 P(AB) < P(u + max |tf 3 (JV)|/B(n) > u). 

N:l/v<r* /B{n)<v^ 

We find analogously to the (Ostrovsky E., Sirota L., 2004): 

P(v + max \V 3 (N)\/B(n) > u) < 

N°/v<N<vN° 



exp — C 



[u-v) r ((nA{n)) r ^ 



f/2 \ Q g l Q g n 



Thus, P(A) < 



(u-v) r {{nA{n)) r ^ 



exp I -C ^ — I + exp -Cv r 



f/2 \ g log n 



{(nA(n)y/ r 
log log n 
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Choosing v = 0.5-u for sufficiently great values u, u > C, we arrive to the 
assertion of theorem R(q). 

Remark 1. Let us note, and use it below, a slight difference in the behaviors of 
the values r(n,N) and N(n) which consists in the peculiarity of condition (t>). At 
v > 1 we have (under the same conditions (Rq), (v) : 

max (p (^L < I) P >v \\< exp ( Cl AnA(n))^ \ 

maX [ P [N°(n) -v)' P \N°(n) > v ))^^{ Cv (log log n) ) ' 

An analogous estimation for the probability P(r*(n)/B(n) > v) holds even without 
condition (v). 

Remark 2. The consistency of the proposed estimations in the above- 
mentioned sense under all the introduced conditions, including (v), it follows from 
the assertions already proved. Indeed, since 

A(n) < A(n, [«y/ra]) < Cn' 1 ' 2 + -> 0, 

then N°(n) — > oo, N°(n)/n — > 0, because otherwise the value 

A(n) = A{n, N°{n)) x N°{n)/n + p(N°) 

would not tend to zero. 

Since N(n)/N°(n) — > 1, then N(n) — > oo and analogously N(n)/n — > 0, which 
proves the consistency of /. 

We proceed now to the problem of estimating density (D). Here 

c k = L k (x) f(x)dx = EL fc (&), c fc = n _1 ^L fc (&), 
J- 1 i=i 

N(n) n N(n) 

K x ) = L k( x ) = ^E E L fc(a:)- 

fc=0 1=1 fe=0 

The functional r may be written as r(n, iV) = 

n 2N 2N 

^E™ -1 E ^te) + 2^ 2 E E 

i=l k=N+l l<i<j<n k=N+l 

Let us denote 

k 

G k (x,y) = L m (x)L m (y). 

m=0 

It is known (Bateman FL, Erdelyi A., 1953) [chapter 10, section 10], that if x ^ y, 
then 

G k (x,y) = (k + l) [P k+1 (x)P k (y) - P k (x)P k+1 (y)]]/(x - y) 
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and 



G k (y,y) = (k + 1) [P k (y)Pl +1 (y) - P k+1 (y)Pi(y) 
Also 

(l - x 2 ) Pl(x) = k [P fc _i(a;) - xP t (i)] . 
Therefore, we have in the considered problem (D) 

n 

r(n, N) = n- 1 £ n" 1 (GW(&) - GW(&)) + 
i=i 

l<«<j<n 

To + r i + r 2, Er = t ~ i?(n, JV), and the second (and the first) expression for the 
r, i.e. r 2 is the so-called U statistics. 

We find by direct calculation (as in the case of problem R): 

Var[n] < CB(n,N)/n, Var[r 2 ] < CB(n,N)/n. 

Recall that the U statistic with correspondent sequence of sigma- algebras is 
also a martingale. Using the exponential boundaries for the martingale distribution, 
(Ostrovsky E., 2004), (Korolyuk B.S., Borovskich Yu., 1993) etc., we obtain: 



n>16 



sup P(|ri )2 |/ JVar [n |2 ] > x) < exp (-Car) , a; > 0, 



P(C > x) < exp (-C x) , 

where C = sup niJV |C(n, JV)|, 

r( n , N) = f Er(n, JV) + ((n, N) log log n yj B(n, N)/n. 

Repeating the considerations of (Ostrovsky E., Sirota L., 2004] we complete the 
proof. 

5. Adaptive confidence intervals (ACI). Let us now describe the use of our 

results for the construction of ACI. Note first of all that the probability 



P/(«) = P(II/WII 2 > 



If 



with rather weak conditions in all the considered problems permits estimation of 
the form 

P/(u) < 2 exp (-<p{C, n, B(n))u r/2 )) = f P^(u), u>C 1 . (5.1) 
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As it is proved above, the variables Bin), C, C\ have respective consistent estimates, 
for example, 

Bin) ps min rin.N) = r*(n). 

V ' N<n/3 V ' V ' 

The values C, C\ also depends on 7 and on the constants Cj appearing in the defi- 
nition of conditions (7), (v). With very weak conditions they can also be estimated 
consistently by the sampling in the following way. Set M = M(n) = exp( v / logn) ; 
then, if conditions (7), (v) are fulfilled, a system of an asymptotic equalities can be 
written: 

r(M) - a s M/n ~ (1 - i)p(M)\ 

r(2M) - 2a s M/n ~ 7(1 - 7)p(M); 

r(4M) - 4<r s M/n ~ 7 2 (1 - 7)p(M), 

where the symbol s denotes the number of problem. 

Solving this system, we find the consistent (mod P) estimate of 7 : 

r(4M) - 2r(2M) 
7 ~ r(2M) - 2r(M) ' 

(The parameter cr s can also be estimated consistently, but that is not necessary for 
us). The constants C 1 ,C also can be consistent determined. 

Substituting the obtained estimates of all the parameters into (5.1), we get to 
the estimate of the confidence probability 

Pjj-(u) < 2 exp (-0(C( 7 , Ci, C),n, r*(n)) u r/2 ) = f P f (u). (5.2) 

then, equating the right-hand part of (5.2) of the unreliability of the confidence 
interval 5 to, say, the magnitude 0.05 or 0.01, we calculate u = u(5) from the 
relation 

and obtain approximately the adaptive confidence interval for / reliability 1 — S of 
the form 

\\f-f\\ 2 <u(5)rmn 3 r(n,N). (5.3) 

But for a rough estimate of the error from replacing / by / the following quite 
simple method can be recommended. Since 

||/-/|| 2 _ A(n,N(n)) * 3 (N(n)) 
B(n) B(n) B(n) ' 1 ' ' 

and the second term in the right-hand part of (5.4) a.s. tends to zero, while 
the first term, if conditions (7), (v) are fulfilled, has 1/(1 — 7) as its limit, we thus 
prove the following assertion apparently well known to specialists in nonparametric 
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statistics for non-adaptive estimation: 



Theorem c.i. // the following conditions are fulfilled in our problems: in the 
problem R (Rq), (7), (v) or (7), (v) in problems D, S, then 

ImT_ 0O ||/-/|| 2 / J B(n)<l/(l-7). (5.5) 

In order to construct an adaptive confidence interval assertion (5.5) can be re- 
formulated as follows. With probability tending to 1 as n — > 00 

||/-/||*<B(n)/(l-7), (5.6) 
and ACI is constructed by replacing the values B(n), 7 by their consistent estimates: 
II f f\\*<^ t(2M)-2t(M) 

117 " 711 " T (U) 3r(2M)-2r(Af)-r(4M)- ^ 

A more exact result will be obtained by taking into account the following term of 
the expansion of the value || / — f\\ 2 : 

llf-fll 2 1 c 

B{n) 1-7 ^N°(n) 

where e n — ► 0; P(|C| > u) < 2exp(— CV"/ 2 ) and C no longer depends on n. Equating 
the probability P(|C| > u ), more exactly its estimate 2 exp(— Cu r ^ 2 ) to the value 
5, 5 ~ 0+, we will easily find w = u(5) and construct an approximate ACI with 
reliability ~ 1 — 5 of the form 

\\f-f\\ 2 <^ + r*(n)u(5). 

Closer consideration reveals an effect that somewhat reduces the exactness of ACI. 
Let (as is true in all the three considered problems under the formulated assump- 
tions) 



||/ - f\\ 2 /B(n) >u)< exp(-0(C 1 ^), <f>(u) = <f>(n,u), 
P (r*(n)/B(n) < l/u) < exp(-^(C 2 u)), u > C, 
where at u — > 00 =>- 0(it) — > 0. We denote 

Q(u) = P(\\f-f\\ 2 /r*(n)>u). 

Theorem r. At u < C / B{n) the following inequality holds: 

Q{u) < 2exp(-0(CV^)). 
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Proof. We have by the full probability formula we (we will understood P(A/B) as 
the conditional probabilities, if, of course, A and B are events): 



Q(u) < P 



I/-/II 2 
T*(n) 



> u/ 



T*(n) 1 
— > - 
B(n) v 




T*(n) 1 
— ^— f > - 
B(n) v 




+P 




Qi < P (11/ " /|| 2 /S(n) > u/v) < eM-tiCm/v)); 



Q1 + Q2; 



Q 2 <P (T*(n)/B(n) < l/v) < exp (-0(C 2 u)) . 



Summing up and put f = Csv^m, we obtain the assertion of the theorem. 

The increase in the probability Q compared to P/ is apparently explained by 
the ability of the denominator, i.e. r*(n) to take values nearly to zero. 

Note in conclusion that the estimates proposed by us have successfully passed 
experimental tests on problems R, D by simulate of modeled with the use of pseudo- 
random numbers as well as on real data (of financial data) for which our estimations 
of the regression and density were compared with classical estimates obtained by 
the kernel and wavelets estimations method. The precision of estimations proposed 
here is better. 

The advantage of our estimations in comparison to the trigonometrical estima- 
tions [Ostrovsky, Sirota, 2004] is especially in the case when the estimating function 
/(•) is not periodical. 
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